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A 5~week intensive training institute was conducted for 13 public school 
administrators wito have research responsibilities under Title I and Title III projects. 
The program focused on research methodology and design, statistical procedures, 
evaluation of research, and proposal writing. The general format was a lecture or 
presentation period followed by a laboratory session for immediate application or 
study of the materials presented' in- the earlier session. Some time in each session was 
devoted to discussions and to question-and~answer activities, and 3 field trips were 
made to nearby research centers. The ability of the University to provide immediate 
computer application aided the teaching program. Pre- and posttests, measuring 
understanding of research terms and concepts, indicated substantial achievement of 
institute objectives. Participantv evaluations revealed that they got essentially what they 
had expected to get from the institute although they felt a greater length of time was 
needed to permit proper coverage of the proposed curriculum. A high level of 
participant motivation coupled with the enthusiasm and ability of the staff 
strengthened the program. Appended are publicity materials and 1 1 pages of written 
evaluations by the staff and 6 participants. (JSt 
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FINAL REPORT 
OF 

INSTITUTE OF PUBLIC SCHOOL RESEARCHERS 



Bureau of Educational Re'earch 
University of Denver 
June 20 - July 22, 1966 



ORIENTATION OF PROGRAM 

This institute was designed to provide intensive training 
in research for a specific group of public school and state 
department employees. It was a requirement of eligibility that 
the public school employee have a designated responsibility for 
federal programs or research in his school district and that 
state department employees have a primary association with proj- 
ects requiring a knowledge of research methodology. The objec- 
tives of the program were concerned with research methodology 
and design, statistical procedures, evaluation of research, and 
proposal writing. The trainee group consisted of thirteen public 
school employees who were superintendents of schools and other 
administrative employees having responsibility in research under 
Title I and Title III projects. There were no state department 
trainees in attendance. There was a wide variance of background 
in statistics and research methodology among the participants and 
the extent of previous experience. The training program began on 
June 20, 1966 and ended on July 22, 1966. The instructional 
staff attended a three-day pre-planning session and a two-day 
post-evaluation session. 



DESCRIPTION OF THE PROGRAM 

The instruction schedule was carried out in two daily 
sessions of approximately three hours each. The general format 
was a lecture or presentation period followed by a laboratory 
session for immediate application or study of the materials 
presented in the lecture session. In addition to pre and post 
evaluations, examinations on the content materials were adminis- 
tered several times during the session. The laboratory assistant 
and one or more members of the instructional staff were in 
attendance in all laboratory and lecture sessions. Some time in 
each session was devoted to discussions and to question and 
answer activities. 



The major content areas were organized to give considerable 
instruction and experience in research designs through the level 
of CO -variance and factorial experiments. The necessary statis- 
tical computations were included. Most of the standard research 
models were considered and computer adaptations were carried out. 
One week of the afternoon sessions was devoted to the study of 
the preparation of research proposals and their evaluation. 
Finally, considerable attention was given to the proper evalu- 
ation or interpretation of research designs and statistical 
inferences . 

On a topical basis the following items were included in 
the presentations by the staff in the five-week training session: 

Descriptive statistics 

Population and sample 

Ranking procedures 

Validity 

Variability 

The normal curve 

Correlation 

Reliability 

Experimental errors 

Analysis of variance 

Analysis of co -variance 

Factorial analysis 

Probability 

Sampling errors 

Standard errors of the mean 

Standard errors in the difference of means 

Test of significance 

Predictive procedures 

Chi square 

Pre-test and post-test design 

Pre-test and post-test control group design 

Solomon's four group design 

Post- test only control group design 

Time series 

Non-equivalent control group 
Q sort 

Ex post facto research 

Randomized block 

Type I and type II errors 

Computer application 

Problems and delimitation 

Problem construction 

Procedures and findings 

Conclusions and evaluation procedures 

Proposal writing in Titles I and III in Public Law 89-10 
Proposal case studies 
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No major changes were made in the program following the 
approval of the proposal. During the pre-planning period, the 
staff was able to prepare a schedule that dealt in considerable 
depth with each of the principle objectives. There were some 
modifications, particularly in the area of research writing, 
wherein the background of the candidates and the developing 
limitations of time restricted exercise in actual proposal 
writing. After the first Thursday seminar, the participants 
requested that this time be allocated to work in the computer 
lab. It was necessary to spend more time than had originally 
been planned on the use of the calculator during the lab sessions. 
The week devoted to proposal preparation was advanced to the 
second week due to a previous commitment on the part of one of 
the staff members. One staff change was made following the 
approval of the proposal. Dr. George Mouly of the University of 
Miami withdrew when his own application for an institute was 
approved. Mr. Bobby Hopkins proved to be an acceptable 
replacement. 



EVALUATION OF THE PROGRAM 

The following evaluation statements were based on (1) a 
pre-test, a post-test, and unit tests of the understandings of 
various research and statistical terms, (2) evaluative reports 
by the instructional staff, (3) evaluative reports by the trainees, 
and (4) general observations of the program director. It did 
appear that despite the wide range in backgrounds of the trainees 
and some, minor modification in the amount of material that was 
covered, that the principle objectives and goals of the institute 
were met successfully. 

Program factors. There is considerable evidence to 
indicate that the principle objectives of the institute were met. 
The pre-test (patterned after Stanley's AERA approach) sampled 
the level of understanding that the trainees had with regard to 
research terms and concepts. Their understanding level and 
computational ability with regard to statistics as measured on a 
five point scale on the pre-test indicated that the understanding 
level was predominantly at the C, D, and E level for research 
terms and concepts. There was a somewhat higher level of under- 
standing of statistical terms although the level remained 
predominantly C, D, and E. At the computational level, there 
was some indication of higher accomplishment at the lower level 
of descriptive statistics. When the same instrument was applied 
as a post- test, the level of understanding swung sharply to the 
A, B, C level. Some notable exceptions to this improvement were 
related to error terms, rectangular distributions, affective 
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cognitive objectives, hypothetical constructs, and fixed-random 
models in the area of statistics. A low level of understanding 
remained for such terms as kurtosis, Duncan's tests, Fisher s z, 
index of discrimination, and parameter. At the computational 
level, the participants indicated that they were able to perform 
most of the computations required at the end of the institute. 
There remained some computing inability related to validity, 
kurtosis, Duncan's test, Fisher's z, index of discrimination, 
and normalizing a distribution. It is the opinion of the 
director that sufficient time was not available to improve the 
computation ability further. The evaluations submitted by the 
participants indicated that a greater length of time was 
necessary to permit proper coverage of the proposed curriculum. 
The participants, to a large extent, indicated that they got 
essentially what they had expected to get from the institute. 

The content of the institute appeared adequate and proper when 
related to the objectives, the background, and the desires of 
the participants. The major focus on research designs, 
statistical computations related to these designs, and to the 
essentials of the preparation of research reports and research 
proposals gave each participant an opportunity to improve his 
skills and understandings in these areas. The lab experiences 
were particularly helpful in that the participants had 
opportunity for immediate application of the lecture materials 
in practical problems. In some instances, the lab time was 
insufficient. The practice exercises, the computer problems, 
and the use of the computer assisted greatly in the attainment 
of an understanding not held by the trainees prior to the 
institute. Field trips to the downtown IBM Center, to the 
University of Denver Data Processing Department, and to the 
computer lab at the Denver Research Institute were of value to 
the participants. However, it appeared that the visit to the 
computer center was of the greatest interest to the participants 

The number of staff members was very adequate for the 
number of participants in training. It is doubtful that the 
staff would have been adequate if a full complement of thirty 
trainees had been in attendance. Higher attendance would not 
have materially effected the lectures, but would have seri- 
ously hampered the effectiveness of discussion periods. It 
also would have made it impossible to give the participants 
the great amount of one-to-one instruction that was carried out 
in the lab periods. Staff had adequate time for the preparation 
of their assignments and testing materials. The director and 
the research assistant had sufficient time to perform the 
administrative tasks. Secretarial assistance was adequate for 
the use of the director and teaching staff. 

It is difficult to determine if the selection criteria 
were too restrictive since the inquiries may have been limited 
by the late date of approval of the institute. It is possible 
that the requirement of responsibility in the school district 
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reduced the number of appliv^ants. The total number of 
applications was below the expected number. The trainees were 
widely distributed over the geographic area defined in the 
brochure and the distribution of the trainees within the six 
states was broad. Perhaps a wider geographic area and a longer 
announcement time would have increased the number of inquiries 
obtained. No problem arose with relation to the housing of the 
trainees in university-owned apartments and commuting between 
these apartments and the university campus. 

There did not appear to be any major problems related to 
the organization of the institute. There was no indication that 
a different time of year would have been more attractive to the 
trainees. It did appear that five weeks was somewhat short for 
the amount of material that was presented. However, it appeared 
doubtful that the trainees could have devoted additional weeks 
to the institute. The daily schedule was planned in advance so 
that the trainees had a long-range view of the presentations. 

Minor modifications were made from day to day to adjust to the 
rate of progress of the trainees. Perhaps the most difficult 
thing to deal with was the lack of assimilation time since the 
workshop was concentrated and covered major quantities of content 
material daily. The classroom and facilities were adequate for 
the number of trainees who participated. Family housing in 
university apartments was quite satisfactory. 

The budget for salaries and stipends was sufficient. 
However, the estimate of 2.5 dependents for trainees was low. 
Actually the number of dependents per trainees was 3.5. This 
did not become a factor financially since the number of trainees 
was less than expected. Several minor internal line item trans- 
fers were necessary. These transfers totaled $350.00, which was 
obtained from staff salaries and did not cause any major problems. 



Strengths and weaknesses . An adequate appraisal of 
strengths and weaknesses of the institute is difficult of assess- 
ment. Certainly the enthusiasm and eagerness of the trainees 
obviously indicated that the faculty and staff constituted a 
strength. The method of presentation was not unusual and no 
major use was made of spectacular media. However, the ability 
of the university to provide immediate computer application 
strengthened the entire teaching program. The facilities pro- 
vided were not unusual, although adequate. Overall, the out- 
standing features of presentation were reflected in the enthusiasm 
for teaching and the ability to explain exhibited by the staff. 
From the trainees standpoint, a real need for the material pre- 
sented and a motivation to study was exhibited and strengthened 



the institute considerably* As far as weaknesses were concerned, 
none appeared inherent in the institute itself. A variance in the 
backgrounds of the students was expected and did require some 
modification in the irate thAt material was presented. Some of the 
organisational problems arose from the short period of time avail- 
able between the award of the institute and the starting date. 
Overall, it remains the opinion of the institute. director that the 
institute was successfully carried out, that most of the trainees 
benefited considerably,^ and that the institute met their expecta- 
tions. The pre-test and post-test and the evaluation supports 
this opinion. 

Except for the aforementioned lateness of award, no major 
difficulties were encountered in the USOE organization of the 
program. The demands of the proposal and budget were not excessive 
and the instructions for preparation could be followed without 
undue concern. The office was ’ ery cooperative in answering mail 
and telephone inquiries. The USOE staff visitation of the 
institute was welcome and appropriate. 



PROGRAM REPORTS 

1. Publicity 



The institute was publicized by means of newspaper 
releases through the University of Denver Public 
Information office. These releases appeared in the 
Denver Post and the University of Denver Clarion . 

A letter of invitation and a brochure were mailed to 
all superintendents of schools in the eight state 
region devised for the institute. Approximately 300 
brochures were mailed. The first mailing occurred on 
May 6 and a follow-up letter clarifying certain 
questions raised by early respondees went to the same 
superintendents on May 16. 

2 . Application Summary 

a. Approximate number of inquiries from prospective 

trainees (letter or conversation) 23 

b. Number of completed applications received 18 

c. Number of first rank applications (Applicants 

who are well-qualified whether or not they were 
offered admission) 18 

d. How many applicants were offered admission 18 

6 
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3 . Tra'inee Summary 



a. Number of trainees initially accepted in program. 18 
Number of trainees enrolled at the beginning 
of program 13 

Number of trainees who completed program 'Z 



b. Categorization of trainees 

(1) Number of trainees who principally are ele- 
mentary or secondary public school teachers 1 

(2) Number of trainees who are principally 

local public school administrators or 
supervisors 12 

(3) Number of trainees from State education 

groups 0 

(4) Number of trainees from colleges or 

universities, junior colleges, research 
bureaus, etc. 0 

4. Program Director ' s Attendance 

a. What was the number of instructional days 

for the program? 25 



b. What was the per cent of days the director 
was present? 



90 



FINANCIAL SUMMARY 



RESEARCH INSTITUTE 986 



DIRECT COSTS 


BUDGETED 


EXPENDED 


BALANCE 


Personnel 








Faculty 


$5750.00 


$5346.01 


$ 313.99 


Secretary Salary 


1500.00 


1179.75 


320.25 


Graduate Assistant 


1100.00 


1100.00 


- - - 


Books and Periodicals 


400.00 


275.53 


124.47 


Supplies and Expenses 


1200.00 


882.25 


317.75 


Equipment Rental 


600.00 


525.00 


75.00 


Travel 


500.00 


212.50 


287.50 


Staff Fringe Benefits 


810.00 


810.00 


- - - 


Training Stipends 


15750.00 


8194.76 


7555.24 


To ta Is 


$27610.00 


$18615.80 


$8994.20 
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APPENDIX A 



BROCHURE 




; AN INSTITMTE FOR PUBLIC SCHOOL RESEARCHERS 

Bureau of Educational Research 
University of Denver 
June 20 - July 22, 1966 

f The Bureau of Educational Research in the School of Education at the University 
if Denver in cooperation with the U. S. Office of Education offers a five week Insti- 
tute for Public School Researchers from public school districts and state education 
fgencies in the Rocky Mountain Region as authorized under Title IV of the Elementary 
[nd Secondary Education Act of 1965. In conducting this Institute, the University of 
Denver will adhere to Sec. 601, Title VI of the Civil Rights Acts. 



objectives 

r The objectives of the Institute are 
to offer public educational directors 
pf research and special programs spec- 
ialized study in research methodology, 
design, measurement and evaluation, and 
research x^rriting. The purpose of the 
Institute is to train the participants 
in the following areas: 

A. Research methodology . To gain 
an understanding of usable methods in 
the scientific approach to problem- 
solving. To incorporate scientific lan- 
guage and methodology in preparing re- 
search proposals. 

‘ B. Research design . To comprehend 
the meaning, purpose, and principles 
underlying scientific methods in research 
^design. To develop sound, sophisticated, 
and less sophisticated techniques in 
evaluating programs. 

! C. Program evaluation . To afford 
knowledge regarding methods of data col- 
lection and the relation of these meth- 
ods to eventual evaluation. To analyze 
and interpret data in respect to projec- 
ted hypotheses through the use of sta- 
tistical tools on an elementary level. 

D. Research writing . To prepare 
responsible officials to write meaning- 
jful reports of completed research for 
‘use and for appropriate dissemination. 

I A primary objective of the Insti- 
|tute is training educators to write pro- 
fposals and to evaluate programs pertain- 
ing to Titles I, II, and III of the Ele- 
Imentary and Secondary Education Act of 
1 1965. 



pr og ram 

The program of study will follow 
two approaches. First, fundamental re- 
search tools will be taught in a class 
situation adapted to the objectives of 
the Institute and the needs of the par- 
ticipants. Second, statistical tech- 
niques will be adapted to practical 
problems in education through a broad- 
field seminar approach. 

The class curricula will include 
meaning, characteristics, and processes 
of educational measurement and evalu- 
ation; development and interpretation of 
standardized instruments; research proc- 
esses related to educational evaluation; 
and selected statistical tools and proc- 
esses. 

The seminar will be organized into 
the following areas: 

A. Research design . Problems and 
hypotheses. Constructs, variables, and 
definitions. Sampling and randomness. 
Design and application. 

B. Program evaluation . Method- 
ology: theory, structured and unstruc- 
tured Q sorts, analysis of variance, 
correlation, significance. Interpreta- 
tion: statistical presentation and 
analysis, interpretation of research 
data, conclusion. 

C. Research writing . Research 
oriented format for report writing. Con- 
cise, objective, clear presentations. 
Dissemination: methods and media. 

The seminar will meet daily for 
three hours each morning. The afternoon 
session will meet two hours daily. 
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eligibility 

The thirty (30) participants will 
be selected according to the following 
criteria: Ttie participant must 

k, be a public school official 
designated by a local board of education 
as responsible for federal programs or 
research in the school district ^ be a 
State Department of Education official 
whose job responsibilities are primarily 
related to projects requiring research 
methodology. 

B. possess a graduate degree and 
exhibit a previous course work pattern 
which would indicate probable success in 
an intensive institute of this type. 

Selection will be made to insure a 
broad geographic representation from the 
following states: Colorado, New Mexico, 

Wyoming, Utah, western Nebraska, and 
western Kansas. 

credit 

No academic credit will be offered 
for participants in the institute. 

st i pends 

Each participant will receive $75 
for each week of the Institute plus $15 
a week for each family dependent. Trav- 
el allowance will be made in the amount 
of 8d per mile for one round trip. 

staff 

Dr. Ralph A. Forsythe, Assoc. Professor 
University of Denver 
Dr. William M. Slaichert, Professor 
University of Denver 
Dr. George J. Mouly, Professor 
University of Miami 

housing 

The housing facilities at the Uni- 
versity afford single accommodations in 
student residence halls or family accom- 
modations in apartment halls. Complete 
information concerning housing accommo- 
dations and an application for reserva- 
tions are attached to this brochure. 



t he bur eau i 

The Bureau of Educational Research i 
is a division of the University of Denver j 
School of Education. University insti- j 
tutional studies and projects contracted* 
by the School of Education faculty are i 
conducted in the Bureau. Specific proj-| 
ects funded through federal and founda- | 
tion grants are also undertaken in the | 
Bureau. Other studies are conducted 
which deal with finances, enrollment | 
projections, classroom needs, curriculum j 
improvement, special disciplines, and I 
federal programs relating to public! 
school systems. | 

The Bureau of Educational Research! 
is the center for the Rocky Mountain | 
School Study Council. Work and study of | 
six standing committees made up of mem- j 
bers from the eleven metropolitan Denver I 
school districts in the Council are ad-^ 
ministered through the Bureau of Educa-i 
tional Research. ^ 

t he s c hoo I 

The School of Education in the Uni-; 
versity of Denver is committed to thej 
philosophy that teacher education in a= 
private university has both a unique op-^ 
portunity and a fundamental responsibil-j 
ity to be experimentally oriented. This 
responsibility, then, is discharged iri 
terms of organization and flexibility of 
the curriculum and emphasis upon research 
activities. • 

The curricula of the School of Edu- 
cation lead to baccalaureate, master's,J 
and doctoral degrees in a number of 
areas and specializations. The programs 
of teacher education at the University 
of Denver are accredited by the National 
Council for the Accreditation of Teachel 
Education. ; 

i 

J 

For Further Information Write : ■ 

Dr. R. A. Forsythe, Director j 

Institute for Public School Researchers^ 
P. 0. Box 72 

University of Denver ' 

Denver, Colorado 80210 ; 



APPLICATION FOR ADMISSION 



INSTITUTE FOR PUBLIC SCHOOL RESEARCHERS 
Bureau of Educational Research 
University of Denver 
and 

U. S. Department of Health, Education and Welfare 

Office of Education 
Washington, Dc G. 



I. Personal 

Name 

Home 

Address 



Office 

Address 



Social 

Security 



phone phone 

Sex Age Marital status 

Number in family eligible for Institute dependency allowance 

(See attached note for qualification) 

IT. Professional Training - Colleges and Universities Which You Have Attended 
Name of Institution Degree Major Minor Dates Attended 



College and University Course Work 



Quarter 



Course Title Grade Credit Hrs. Year Taken 



Intro. Educational Measurement 








Adv. Educational Measurement 








Introduction to Statistics 








Advanced Statistics 
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Course Title 


Grade 


Quarter 
Credit Hrs. 


Year Taken 


Intro. Research Methods 








Adv. Research Methods or Design 









Other related course workr 



Accumulative Grade Point Average i. graduate study onlvl 
A=4, B=3, C=2 



G.P.A. 



III. Professional Experience - List all professional experience for the 

last ten years. Start with your present or last position and work back. 

■ ■' Employer Position From To 



I certify that the statements made by me in this application are 
true and correct to the best of my knowledge and are made in good faith. 



Signature of Applicant 



Date 



TO BE FILLED OUT BY EMPLOYER 

I hereby certify that £g 

primarily responsible for research and/or federal program development and/or 

evaluation in 

~ ^ • 

educational agency 



signature of employer 



title 



date 



UNIVERSITY OF DENVER 

COLORADO SEMINARY 



Sckool of Education 

University Part, Denver, Colorado 80210 



Dear Colleague: 

Regarding the Institute for Public School Researchers (brochure 
attached) to be conducted in the Bureau of Educational Research at the 
University of Denver, June 20-July 22, 1966, the following dates and 
procedures should be considered. 

1. Deadline for making application for the Institute and housing 
May 18, 1966 

2. Notification of participants selected - before May 27, 1966 

3. Registration and get-acquainted session for participants - 
June 20, 1966, 9 a.m. - noon 

4. Holiday - July 4 

5. Institute ends - July 22, 1966 
The application must include: 

1. Completed application form 

2. Certification by employer that applicant s job responsibilities 
are primarily related to research and/or federal programs 

3. A letter of application which 

a. explains the applicant's job 

b. points to the applicant's need for the training he would 
receive in the Institute 

Housing: If the applicant would like to make reservations for 

university housing, he should return the housing form with the application. 
It is understood that the reservations would be contingent upon attendance 
in the Institute. 

You may address all inquiries regarding the Institute application 

or housing reservations to: Dr. R. A. Forsythe 

Bureau of Educational Research 

P. 0. Box 72 
University of Denver 
Denver, Colorado 80210 
phone: 753-2300 or 753-2516 
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HOUSING INFORMATION 



INSTITUTE FOR PUBLIC SCHOOL RESEARCHERS 
Bureau of Educational Research 
University of Denver 
June 20 - July 22, 1966 



Students and faculty members who are on the University of Denver 
lapus for all conferences, workshops and institutes, regardless of period 
: time housing is needed, are assigned to the CENTENNIAL CONFEi^NCE CENTER, 
lis facility is available to single women, single men and married couples. 
)artment facilities are available upon request. 



5 NTENNIAL CONFERENCE CENTER RATES 

The following rates are American plan (room and three meals per days) , 
11 linens, maid service, telephones are provided. Valet service is available, 



Academic Rate - for 

Daily 

Weekly 



Institute participants 
double room 
$8.00 per person 
$48.00 rer person 



single room 
$10.00 per person 
$60.00 per person 



PARIMENT FACI LITIES 

f Furniture, dishes, and utilities only are furnished j linens are 

■vailable at an added cost. The number of apartments available on campus is 
Imited. Reservations will be made on a first-come basis. 



By the Month 1 bedroom - $100.00 

2 bedroom - $117.00 



Institute participants may request reservations in the Centennial 
ionference Center or University apartments or they may make their own arrange- 
nents for accommodations off campus. 

: 



f' 



{ 

( 
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RESERVATIONS FOR HOUSING 




INSTITUTE FOR PUBLIC SCHOOL RESEARCHERS 
Bureau of Educational Research 
University of Denver 
June 20 - July 22 , 1966 

Check one. Reservations will be made for the duration of the Institute. 

Reserve accommodations for me in a single room. 

Reserve accommodations for me in a double room. 

Reserve a double room for myself and my wife (twice douTjleroom 

rate) . 

Reserve a one-bedroom apartment for me. 

Reserve a two-bedroom apartment for me. 

Do not reserve accommodations for me. 

Return this form with the Institute application. It is understood 
that the requested reservation for housing is contingent upon acceptance 
and attendance in the Institute. 

signature 

printed name 

addre s s 

phone 
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STAFF EVALUATIONS OF THE 
INSTITUTE FOR PUBLIC 
SCHOOL RESEARCHERS 



William Slaichert 

The evaluation of an institute of this nature is somewhat 
difficult since most impressions can be gained by observed behavior 
and limited objective evidence. I will, therefore, give my 
impressions based on discussions with the participants and the 
observed behaviors of them- Five major aspects of the program 
will be evaluated including: goals, physical environment, staff 

personnel, participants, and instruction. 



GOALS 



It would appear that most of the major objectives of the 
Institute were reached. The participants seemed to have the 
facility for discussing appropriate research designs with proper 
interpretation and analysis. In the weekly evaluations, they 
demonstrated a rather thorough understanding of the objectives 
that were taught. It would appear that very limited information 
was obtained as to their ability to write research proposals. 
This could be included in a future institute. 



PHYSICAL ENVIRONMENT 

Fortunately, the enrollment did not live up to expectations 
and the room size for the number of participants was adequate. 

The room would not hold more than two or three more and still make 
instruction possible at the efficient level that was had in this 
Institute. It would appear that the maximum size for the Insti- 
tute should be limited to twenty people. This provides for a 
proper interaction between the instructional staff and the 
participants . 

The use of the high-speed calculators made it possible for 
the participants to engage in the computational aspects of 
statistics at a rather efficient pace. It would appear that the 
ratio of two participants per machine was adequate. In spite of 



the malfunction of two machines, effective service was provided 
by the Monroe Company in servicing on an immediate basis. One 
recommendation that would seem appropriate was that future insti- 
tutes be housed in a facility that had adequate blackboard space. 
As the statistical analyses became more suitable and complex, 
more blackboard space was needed to develop the overall analysis. 
The participants seemed to indicate that the facilities were quite 
adequate and provided a proper setting for this kind of institute. 
All library materials were immediately available and the partici- 
pants seemed to make effective use of them. 



PERSONNEL 

Excellent help was provided by the secretaries involved in 
this Institute. It would appear that it is imperative that at 
least one secretary be provided for each staff member involved. 

It was possible to provide resource materials for the participants 
almost immediately. 

Under the able direction of Dr. Forsythe, the Institute was 
well organized and integrated. No problems were evident, which 
made the Institute a smooth operation. Under the direction of the 
director, all anticipated problems were discussed before the 
Institute was in session and during the Institute a number of 
staff meetings were held to make sure that no problems occurred. 

A great deal of credit is due to Dr. Forsythe for the success of 
this Institute. The graduate assistant, Mr. Jerry Southard, 
provided able assistance during the regular teaching session as 
well as providing the added instruction necessary during the 
laboratory sessions. Each participant was able to get his 
individual questions answered without delay. 



PARTICIPANTS 

The participants of this Institute were quite unusual. 

They represented a highly motivated and a cohesive group after the 
first few days. Basically, they represented an unsophisticated 
research group but were so highly interested in learning that 
rather rapid progress could be made during the instructional 
sessions. It should be noted that they refrained from discussing 
personal experiences. This seemed so unusual from other workshops 
or conferences in which I have been involved. They made valuable 
use of the lab time that was provided for them. In fact, many 
times they had to be asked to leave since the time had been used. 
They were able to relate the instructional experiences to an 
applied situation in a school district. They were primarily 
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interested in how they could improve the research program for public 
education. This is a credit to the participants. As the Institute 
progressed, it was observed that they were becoming sensitized to 
the problems that a researcher faces in an applied setting. 



INSTRUCTION 

One small problem was experienced due to a prior committment 
of this instructor. The statistics aspect was not interlocked with 
materials that were presented during the "research design" sessions. 
However, Mr. Hopkins did provide sufficient background whereby the 
participants could interpret the designs that he was giving. 

Within a short time when the statistics sessions had caught up with 
the research designs sessions, the participants were able to 
integrate the material. It would seem that in a future institute, 
the statistics might be taught on a full-time basis during the 
first two weeks up through simple analysis of variance. This 
would provide the framework for participants to understand many of 
the basic designs handled during the research sessions. However, 
during the fourth week of the Institute all of the participants 
indicated that they were understanding all materials covered and 
the "interlocking" problem had been solved. The textbook used 
for the sta titles was adequate for the first two weeks. However, 
it would be suggested that funds be provided to buy a second 
statistics book which would stress those materials necessary for 
analyzing more complex designs in educational research. I think 
this Institute demonstrated that it is possible to teach sophis- 
ticated statistical analyses providing the elem.entary statistics 
are well understood. 



GENERAL OVERVIEW 

I believe this Institute demonstrated that is is possible 
to attain a high level competence in statistical analysis pro- 
viding a highly motivated and interested group of participants 
are involved. 

It is recommended that a follow-up should be made in regard 
to these participants in order to determine if the material they 
had obtained during this Institute was being applied in their 
local situations. This could be done either through visitations 
to their local school by consultants, or by holding a week-long 
institute during March or April of 1967. 

It is recommended that a future institute of this type 
be limited to twenty participants who will be responsible for 
research programs in their school districts. 
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It seemed evident in this Institute that the participants 
lacked background in the multitude of evaluative instructions 
that might be used in research projects. It would seem appro- 
priate that an institute could be held which would describe and 
help develop evaluative instruments including those for school 
surveys, behavior changes, and attitudinal aspects of educational 
research. 



Bobby Hopkins 

A.' Physical aspects. The classroom facilities, equip- 
ment, and supplies were satisfactory. The personnel and staff 
were cooperative and congenial, and the secretarial assistance 
provided was excellent. The computer facilities which were made 
available to us were adequate. The employees at the computer 
center gave excellent and prompt assistance. The integration of 
the research design class and the statistics class could have been 
improved. More time should be devoted to the research design 
principles toward the end of the Institute. The instruction 
concerning proposal writing should be an integral part of the 
Institute rather than being presented in large doses for a period 
of one week. In addition, each participant should be involved 
in writing an actual proposal which might possibly be applied 
for his own school district. 

The Thursday evening seminars were not of value possibly 
due to the fact that with this small group of participants the 
necessary questioning and discussion took place in the class 
sessions themselves. However, I personally doubt whether such 
an evening seminar would be of value even for larger groups. A 
weekly seminar which could be held during the day instead of one 
of the regular class sessions would more likely meet a need. 

Galfo and Miller's text was suitable for a group at this 
level of sophistication as was the statistics handbook by McKnight 
and McKnight. Unfortunately, the Campbell -Stanley chapter in 
Gage's handbook was out of print and consequently we were unable 
to provide each participant with a copy. This was undesirable 
but unavoidable for this Institute. Hopefully, this will not be 
repeated. 



B. Participants. The participants showed a real interest 
in the object material which was presented. It was evident that 
each one put forth a reasonable amount of effort in order to 
increase his own understanding of the principles and procedures 
presented. I feel that the most significant impact of the Insti- 
tute is revealed by their acquired attitude toward research as a 
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method of securing knowledge. I am sure that each participant 
increased his skill and understanding of the statistics involved 
in research as well as the experimental design considerings 
involved. However, obviously the participants remain at a rather 
elementary level as research personnel. The five-week Institute 
was of necessity a cookbook approach so that more material could 
be presented. 

It is hoped that the participants gained a sensitization 
to the considerations underlying educational research and that 
these men will follow-up and continue studying in order to 
further prepare themselves for more competent research. I feel 
that the Institute met reasonably well the objectives for which 
it was conducted. 



APPENDIX C 



PARTICIPANT EVALUATIONS 




AN INSTITUTE FOR PUBLIC SCHOOL RESEARCHERS 
PARTICIPANT EVALUATIONS 



Please respond to the following questions either in writing 
or on a dictating machine. 

I. Regarding program of study. 

A. What were the strengths of the program? 

B. What were the weaknesses of the program? 

C. How would you improve the program? 

II. Regarding organization and administration. 

A. Should the Institute be shorter (or longer)? 

B. Should it be at a different time of the year? 

C. Were housing accommodations satisfactory? 

D. Did you have adequate materials? 

E. Other suggestions. 

III. Expectations. 

A. What did you expect to get out of the Institute before 
you came? 

B. Were your expectations reached? 



Jack Burr, Superintendent of Schools, Manti, Utah 



A. Some of the strengths of the program are: 

1. Stimulating an interest in research as an aid to 
efficient educational administration. 

2. Obtaining an understanding of research design and 
methods by which more precision and power can be 
built into research proposals. 

3. Getting a basic understanding of methods of statis- 
tical treatment for research evaluation» 

4. Acquiring some basic tools to use in the evaluation 
of research as reported in the literature. 

5. An awareness of the usefulness of the computer as an 
aid to statistical treatment of data. 

6. A familiarity with the processing of data for computer 
treatment . 

B. Weaknesses of program. 

1. Not enough time to study, 

C. I would suggest a longer time with a team of at least 
three (3) (someone from central office, a principal and 
counselor) so that there could be a better chance for 
implementation when we return, 

II. 

A. I would suggest at least an eight (8) week program to 
study the same amount of material, giving participants 
more time for reading, 

B. Summer is the ideal time, 

C. Housing was very satisfactory. 
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D. I had adequate materials available but not enough time 
to study them. 

E. Have more of them and bring teams from participating 
dis tricts . 

III. 

A. Expectations - The brochure advertising the Institute gave 
a pretty good description of what actually took place. 



Kenneth Gardner, Superintendent of Schools, Falls City, Nebraska 



I. This is a difficult program to evaluate due to my lack of 
knowledge at the beginning. However, to begin with, I want to 
say that this has been a most valuable five weeks, and I feel that 
the amount of material and information and education received is 
(Yiore than I can evaluate in these few moments « I feel one of the 
main strengths of the entire program was the staff that was avail- 
able to the Institute. I certainly want to thank all who were 
participating: Dr. Forsythe, Dr. Slaichert, Mr. Hopkins, and 

Jerry Southard. This is the kind of confidence that I gained 
from this five weeks of the Institute. The entire staff has been 
more than cooperative and certainly had a good understanding of 
our problems from the beginning. I feel that with the extra time 
that they have put in to make this a worthwhile program certainly 
speaks well of staff here at the University as well as of the 
Institute. Also I would like to say that the attitude of the 
entire University staff has been most heartwarming and certainly 
they have made every effort to make this a helpful and educational 
project. Also, to the staff of the Bureau here who has also 
helped to make it a worthwhile stay. The availability of the 
computer and the room to punch the cards certainly was a thing 
that I had not expected. This has been most valuable and I feel 
will be able to be made use of in my own school system. The 
size of the class was a particular strength in that we were small 
enough that this allowed us to make use of the machines in the 
computer area as well as to ask questions and to seek the advice 
of the staff members. I certainly would recommend the small 
class in another institute. I would judge the entire program as 
a strong program for an institute of this type and I feel that it 
was well organized. 

I was unable to list any particular strong weaknesses of 
the program since it was so well organized. About the only weak- 
ness I would say was the lack of blackboard space for Dr. Slaichert. 
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II. Probably the Institute could have been longer in 
terms of the number of weeks. I thought that possibly eight weeks 
would have been a greater advantage to the type of personnel that 
was here. Certainly we covered a lot of territory in five weeks. 
It definately should be in the summer due to the fact that this 
would be about the only time that the personnel could get away. 

The housing was more than adequate and certainly was satisfactory. 
We had adequate materials and certainly anything we wanted was 
accessible to us. I thought that the Institute itself was very 
enjoyable and I and my family want to thank the University of 
Denver, the Bureau of Educational Research, and Dr. Forsythe for 
making this Institute possible. 



I might make one suggestion. That would be if in future 
years, if it is known that there is going to be an institute, if 
some advanced material could be sent out to the participants so 
that they could get some background of what they might cover 
during the institute it would be helpful not only to the pro- 
fessors but also to the students who would be participating. 



III. I felt that I had a very small background in 
statistics and in research, and I felt that this is going to be 
necessary in the future for all programs concerning the public 
school. If there were any inadequacies, it was on my part. 
Certainly the amount of material and the information in the 
material presented us would be useful in my own situation and I 
hope to make use of the various designs and methods of research 
and statistics that v?ere presented to us here. I would say that 
my expectations were reached to the highest degree. Again, 
thanks to all who made our stay in Denver and at the University 
so delightful. 



Edward DeJarnett, Coordinator of Evaluation, Hobbs, New Mexico 

I. Regarding program of study. 

A. Strengths of the program were three things combined. 

1. Men interested in learning and teaching without the 
grade effect. 

2. Interaction of the students and professors on their 
problems and attempts to solve them. 

3. High caliber type of teachers and Graduate ass is rant. 
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Bo Weakness of program. I see no weakness other than I could 
have appreciated more time in the library to check out 
some of the studies mentioned and crossed references on 
some of our problems within our local system. 

C. Improvement of program. If the Institute could afford 
three people from each system (example; one teacher, one 
counselor or worker in statistics, and one central office 
administrator), the program could be set off in the local 
school system with three people to support the ideas 
gleaned from the workshop. 

II. Regarding organization and administration. 

A. Time was just right. 

B. Summer was o.k. 

C. Housing was very satisfactory. 

D. Materials were satisfactory. 

E. Follow up with what we are doing might be highly beneficial. 

III. Expectations. 

A. Proposal writing, statistics, advice on designs, research. 

B. Expectations were reached. I could have used more on 
proposal writing and examples of good types of proposals. 
The part we received was great. I could have had a whole 
summer of this type of program. I'm sure my school system 
will be pleased. 

Summary, I was most impressed with your University. The staff 
was cooperative in helping me work for my school system while 
here. I received much help for the Education Department, Educa- 
tional TV; the museum projects and the men were most helpful. 

The high morale of your staff and within the various departments 
is quite impressive. Your secretarial staff is great. The work 
turnover is high in quantity as well as quality. Thanks so much 
for allowing me to attend. 




Warren Bostrom, Teacher, Colorado Springs, Colorado 



I. I thoroughly enjoyed the research Institute. I feel I 
have a greater grasp of statistics and research design than I 
have ever had before. I consider the strengths of the program 
to include such areas as the small group. I feel that had we had 
more than the thirteen, we would not have received nearly as much 
benefit as we have. Our teachers, including Mr. Hopkins, Dr. 
Slaichert, Dr. Forsythe, and Jerry Southard, have been excellent. 
Such minor items as name plates in the front of the various 
participants was good. I feel that I do know considerable more 
than I did on how to set up an experimental unit, how to write a 
proposal, how to find material as far as researches are concerned, 
I have a much better understanding of the computer, I understand 
the roll of the calculator as far as statistics are concerned, 
and I feel that I have a sensitivity to the whole area. As yet, 

I still do not feel real confident in the overall picture but 
through notes that I have taken, through books that I have been 
referred to, and through knowing individuals to whom I can go for 
assistance, I feel definately that I can help set up not only 
programs in my own classroom but also in the school district. 

Even the aspect of no grades was good. We came here to learn and 
did not have the pressure of having to get grades was a good item. 

Again as I have said before, I reiterate that it was good 
overall. However, since nothing is perfect from my particular 
view, I feel that there could have been greater coordination 
between Mr. Hopkins and Dr. Slaichert as far as what each other 
was teaching. I feel, too, that possibly IBM could have been 
better prepared as far as what we had covered in our classroom. 
Perhaps greater coordination, again, between the Institute and 
IBM. We were given a wealth of material during these five weeks 
but how to apply, again, is not quite clear in every aspect, I 
think greater emphasis should be put on application, A sugges- 
tion that has been followed by the Industrial College of Armed 
Forces in their seminars — an outline of the lessons beforehand 
given to each participant might have been of value. It was 
rather difficult to follow the instructors when you were writing 
perhaps considerably behind while they were talking, they were 
putting work on the board, and trying to get this copied down and 
at the same time understanding what they had to say was rather 
difficult. Perhaps an overhead projector with transparencies of 
the various problems and various demonstrations would have been 
good and given handouts to the participants so that they could 
follow each step along the way without having to write it down. 
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I do feel that perhaps more on the roll in application 
statistics would have been good. A Dr. John Starit, who is in 
charge of computers at NORAD, is excellent. He might be a 
welcome addition as a guest speaker at a future seminar. It 
would have been interesting to have had a tour of the DU computer 
center. I would have liked to have had more writing as far as 
designed proposals are concerned. Perhaps more background material 
could be sent the participants before they come to an institute 
so they would be better prepared for what is before them. I feel 
a refresher course, perhaps in March as is tentatively planned, 
would be excellent to firm up the materials that we have learned 
after we have actually put them into operation. 

II. I believe the Institute is just right in length. I 
believe it is the proper time of year. Housing accommodations 
were satisfactory. I believe we did have adequate materials. 

III. What did you expect to get out of the Institute before 
you came is a question that did concern me. I am chairman of the 
Curriculum Committee for Social Studies at the junior high level 
in Colorado Springs. I am also a member of the K-12 Curriculum 
Committee for Social Studies. We talk and sometimes have pilot 
programs going on and I felt perhaps there is inadequacy in 
p3_gnning and control in evaluation. I have learned a number of 
items here which will help in this particular situation. I 
believe my expectations were reached and has opened new windows 
to new worlds. Exactly how they will fall in place still remains 
to be seen, but I do feel we had excellent personnel instructing 
us, taking charge, and I do appreciate and thank those who have 
made this possible. 



I 






Lee Johnsonbaugh, Director of Education, Lusk, Wyoming 



I. Regarding program of study. 

A. It appears to me that the biggest strength of the program 
was the caliber of the instruction. All the instructors 
were very patient, very thorough in their explanations to 
many of us who had not been through some of the materials, 
or if we had, it had been a considerable time since we 
had. The small size of the class led to much group 
discussion, which would not be possible with a larger 
group. Another strength of the program was the time- 
liness relative to the various programs which are now 
getting underway in schools throughout the nation. 
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B. Any ^-^eakness of the program would probably be due to the 
concentrated work which did not really allow enough time 
to consider and digest from one topic to another. 

C. Therefore, the main object or method of improvement 
probably would be a little more time or even a break of 
a week or two somewhere in the middle of such a program, 
if this were possible and probably not, feasible. Many 
of us felt that a more detailed outline of the program 
for the entire time might have helped us a little in some 
of our preparations for the study. Basically, it seems 
that the topics covered were very pertinent to the 
objectives of the whole program. 

II. Regarding organization and administration. 

A. The length of this Institute is probably a little short 
for the materials covered for most of the participants. 
Obviously, the materials could take up to a year or so; 
however, for our purpose in the public schools, the values 
of even the five-week Institute should be very great. 

B. As far as time of year is concerned, the summer is 
undoubtedly the best time for most public school employees. 

C. As far as housing accommodations, we felt they were very 
satisfactory. The apartments were clean and well kept, 
and very adequately furnished. 

D. We felt that we had sufficient and adequate materials 
for our use in the Institute. 

E. Some value would undoubtedly be secured for the partici- 
pants for a follow-up meeting of some kind if it can be 
arranged at a time when participants could attend. It 
was undoubtedly somewhat of a problem to the participants 
in the fact that much of the statistical discussion did 
not precede the presentation of the application of this 
statistical material. This, of course, means that much 
of the time we did not have too much understanding of the 
materials necessary for the applications. 

III. Expectations. 

A. Because of the lack of information, which was undoubtedly 
necessitated by the short time of preparation of the 
director of the Institute, we did not have too much infor- 
mation as to just what the Institute would cover. 
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B. However, I feel that the materials covered should be of 
great value to my work in a small school system. While 
we will probably not do much research as such, it will 
probably give me a considerably better understanding of 
terms and materials written concerning research. 

I have enjoyed very much the five weeks spent at this Institute 
at the University of Denver. I am sure that all the members of the 
Institute appreciated the rather unlimited use of the computer 
facilities, V7hich were new to most of us. However, it would appear 
that more time might have been spent on interpretation of any 
results which we had obtained from computer print-outs. 



Millard Meredith, Elementary Principal, Sheridan, Wyoming 



lo The strength of the program would be the planning and the 
very fine instructional procedures that were evident, and not only 
the instruction during the class time, but the interest and the 
willingness to give help before and after, etc. 

If I were going to list a weakness of the program, I would 
as far as myself is concerned with and this is probably related 
to my background with regard to the subject being worked on, it 
just seemed to me that it was too much material for the amount of 
time. I realize that some basic information would have helped me 
a great deal. However, I do feel much more confident in this area 
and do not really have to have the "fear" that once was very 
evident as far as this area is concerned. 

Improvement of the program--! think probably a longer time. 
However, I have to qualify this for the fact that probably most 
of us, or at least myself, would not be able to be away for a 
longer period of time. Certainly a follow-up and perhaps a 
follow-up in a specific area since we have had what I would con- 
sider a tremendous survey of this advanced field perhaps a follow- 
up in more of a specific area and perhaps this could be related to 
something more personal with each of us. Follow-up procedure-- 
maybe a long week and a full week as has been discussed if we can 
use this Institute or the background of the Institute for a 
resource center so to speak, I would certainly look forward to 
this type of possibility. 

II, I think I have touched on the shorter or longer aspect-- 
I would say longer or less material. I do not believe we can 
change the time of the year, although it could be considered and 
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we might be able to release another time--summer seems to be the 
most satisfactory. Housing I would consider very satisfactory. 

The University of Denver was most agreeable. I have been in quite 
a number of student housings around and I think it was probably 
the nicest or the best accommodations that we have had. Materials- 
I seemed to have had everything that was necessary and certainly 
the handouts and the extra measure that went into giving us the 
handouts, etc. were most appreciated. 

I think perhaps I personally expected a little more in the 
areas of proposal writings, techniques, etc. However, I realize 
that this would come about through probably practice and at the 
same time the evaluation of these programs seems to be the crit- 
ical aspect of the program at the present time, so maybe this was 
what we needed. I did get out of this just about what I expected. 

I think probably the opportunity to have instruction and an 
opportunity to work with the IBM or the computer program in giving 
us an inside or at least a bare beginning on what this tremendous 
potential is for this instrument and hoxfl, if we are going to be 
remaining in this area, that we have got to make use of this 
machine. I think if we do not, we are going to be even farther 
behind comparable other areas. I have appreciated all of the 
courtesies extended to myself in regard to the Institute and I 
would close by saying thank you. 
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